1. Field of the Invention
The invention relates to a holographic display with which voice and holographic image over internet protocol (VHIOIP) services or communications are provided.
2. Technical Background
Computer-generated video holograms (CGHs) are encoded in one or more spatial light modulators (SLMs); the SLMs may include electrically or optically controllable cells. The cells modulate the amplitude and/or phase of light by encoding hologram values corresponding to a video-hologram. The CGH may be calculated e.g. by coherent ray tracing, by simulating the interference between light reflected by the scene and a reference wave, or by Fourier or Fresnel transforms. An ideal SLM would be capable of representing arbitrary complex-valued numbers, i.e. of separately controlling the amplitude and the phase of an incoming light wave. However, a typical SLM controls only one property, either amplitude or phase, with the undesirable side effect of also affecting the other property. There are different ways to modulate the light in amplitude or phase, e.g. electrically addressed liquid crystal SLM, optically addressed liquid crystal SLM, magneto optical SLM, micro mirror devices or acousto-optic modulators. The modulation of the light may be spatially continuous or composed of individually addressable cells, one-dimensionally or two-dimensionally arranged, binary, multi-level or continuous.
In the present document, the term “encoding” denotes the way in which regions of a spatial light modulator are supplied with control values to encode a hologram so that a 3D-scene can be reconstructed from the SLM.
In contrast to purely auto-stereoscopic displays, with video holograms an observer sees an optical reconstruction of a light wave front of a three-dimensional scene. The 3D-scene is reconstructed in a space that stretches between the eyes of an observer and the spatial light modulator (SLM). The SLM can also be encoded with video holograms such that the observer sees objects of a reconstructed three-dimensional scene in front of the SLM and other objects on or behind the SLM.
The cells of the spatial light modulator are preferably transmissive cells which are passed through by light, the rays of which are capable of generating interference at least at a defined position and over a coherence length of a few millimeters or more. This allows holographic reconstruction with an adequate resolution in at least one dimension. This kind of light will be referred to as ‘sufficiently coherent light’.
In order to ensure sufficient temporal coherence, the spectrum of the light emitted by the light source must be limited to an adequately narrow wavelength range, i.e. it must be near-monochromatic. The spectral bandwidth of high-brightness light emitting diodes (LEDs) is sufficiently narrow to ensure temporal coherence for holographic reconstruction. The diffraction angle at the SLM is proportional to the wavelength, which means that only a monochromatic source will lead to a sharp reconstruction of object points. A broadened spectrum will lead to broadened object points and smeared object reconstructions. The spectrum of a laser source can be regarded as monochromatic. The spectral line width of a LED is sufficiently narrow to facilitate good reconstructions.
Spatial coherence relates to the lateral extent of the light source. Conventional light sources, like LEDs or Cold Cathode Fluorescent Lamps (CCFLs), can also meet these requirements if they radiate light through an adequately narrow aperture. Light from a laser source can be regarded as emanating from a point source within diffraction limits and, depending on the modal purity, leads to a sharp reconstruction of the object, i.e. each object point is reconstructed as a point within diffraction limits.
Light from a spatially incoherent source is laterally extended and causes a smearing of the reconstructed object. The amount of smearing is given by the broadened size of an object point reconstructed at a given position. In order to use a spatially incoherent source for hologram reconstruction, a trade-off has to be found between brightness and limiting the lateral extent of the source with an aperture. The smaller the light source, the better is its spatial coherence.
A line light source can be considered to be a point light source if seen from a right angle to its longitudinal extension. Light waves can thus propagate coherently in that direction, but incoherently in all other directions.
In general, a hologram reconstructs a scene holographically by coherent superposition of waves in the horizontal and the vertical directions. Such a video hologram is called a full-parallax hologram. The reconstructed object can be viewed with motion parallax in the horizontal and the vertical directions, like a real object. However, a large viewing angle requires high resolution in both the horizontal and the vertical direction of the SLM.
Often, the requirements on the SLM are lessened by restriction to a horizontal-parallax-only (HPO) hologram. The holographic reconstruction takes place only in the horizontal direction, whereas there is no holographic reconstruction in the vertical direction. This results in a reconstructed object with horizontal motion parallax. The perspective view does not change upon vertical motion. A HPO hologram requires less resolution of the SLM in the vertical direction than a full-parallax hologram. A vertical-parallax-only (VPO) hologram is also possible but uncommon. The holographic reconstruction occurs only in the vertical direction and results in a reconstructed object with vertical motion parallax. There is no motion parallax in the horizontal direction. The different perspective views for the left eye and right eye have to be created separately.
Real-time calculation of holograms requires great computational performance, which can be realised presently for example with the help of expensive, specially made hardware with Field Programmable Gate Arrays (FPGAs), full custom ICs, or Application Specific Integrated Circuits (ASICs), or by using multiple central processing units (CPUs) which are capable of parallel processing.
In thin film transistor (TFT) displays, the pixel pitch in orthogonal directions determines the area per pixel. This area is divided into the transparent electrode for liquid crystal (LC) control, the TFT together with the capacitor and the column and row wires. The required frequency on the column wires and the display dimensions define the required profile and thus the width of the row and column wires.
Ideal holographic displays require a much higher resolution than commercially available TFT-based monitor devices provide today. The higher the resolution, the smaller is the pixel pitch, while the frequency on the row and column wires increases due to the higher number of rows. This in turn causes the proportion of the area covered by row and column wires of the entire pixel area to grow superproportionately compared with the increase in resolution. Consequently, there is much less space available for the transparent electrode, so that the transmittance of the display will drop significantly. This means that ideal high-resolution holographic displays with a high refresh rate can only be produced with severe restrictions. Due to the extreme demands made on the computational performance, the hardware which can be used today for real-time calculation of holograms is very expensive, irrespective of which particular type of hardware is used. Because of the great amount of data involved, the transfer of image data from the computing unit to the display is also very difficult.
A common construction of an active matrix liquid crystal display device will be briefly explained, with reference to prior art FIG. 10 taken from U.S. Pat. No. 6,153,893; U.S. Pat. No. 6,153,893 is incorporated herein in its entirety by reference. As shown in FIG. 10, this active matrix display device has a flat panel structure comprising a main substrate 101, an opposed substrate 102 and a space 103 affixing the main substrate to the opposed substrate, and liquid crystal material is held between the two substrates. On the surface of the main substrate are formed a display part 106 consisting of pixel electrodes 104 and switching devices 105 for driving the pixel electrodes 104 arranged in a matrix, and peripheral driving parts 107 connected to the display part 106. The switching devices 105 consist of thin film transistors. Thin film transistors are also formed in the peripheral parts 107 as circuit elements.
Document WO 2006/066906 filed by the applicant, which is incorporated by reference, describes a method for computing computer-generated video holograms. According to that method, objects with complex amplitude values of a three-dimensional scene are assigned to matrix dots of parallel virtual section layers such that for each section layer an individual object data set is defined with discrete amplitude values in matrix dots, and a holographic encoding for a spatial light modulator of a hologram display is computed from the image data sets.
According to publication WO 2008/025839 of the applicant, which is incorporated by reference, the following steps are carried out aided by a computer:                A diffraction image is computed in the form of a separate two-dimensional distribution of wave fields for an observer plane, which is situated at a finite distance and parallel to the section layers, from each object data set of each tomographic scene section, where the wave fields of all sections are computed for at least one common virtual observer window which is situated in the observer plane near the eyes of an observer, the area of said observer window being reduced compared with the video hologram;        The computed distributions of all section layers are added to define an aggregated wave field for the observer window in a data set which is referenced in relation to the observer plane;        The reference data set is transformed into a hologram plane, which is situated at a finite distance and parallel to the reference plane, so as to create a hologram data set for an aggregated computer-generated hologram of the scene, where the spatial light modulator is disposed in the hologram plane, and where the scene is reconstructed in the space in front of the observer eyes with the help of said spatial light modulator after encoding.        
The methods and displays mentioned above are based on the idea not to reconstruct the object of the scene itself, but to reconstruct in one or multiple virtual observer windows the wave front which would be emitted by the object.
The observer can watch the scene through the virtual observer windows. The virtual observer windows cover the pupils of the observer eyes and can be tracked to the actual observer position with the help of known position detection and tracking systems. A virtual, frustum-shaped reconstruction space stretches between the spatial light modulator of the hologram display and the observer windows, where the SLM represents the base and the observer window the top of the frustum. If the observer windows are very small, the frustum can be approximated as a pyramid. The observer looks though the virtual observer windows towards the display and receives in the observer window the wave front which represents the scene. Due to the large number of necessary transformations, the holographic encoding process causes great computational load. Real-time encoding would require costly high-performance computing units.
Filing WO 2008/025839 of the applicant discloses a method which allows one to generate video holograms from three-dimensional image data with depth information in real time. This makes it possible to generate these holograms using relatively simple and inexpensive computing units.
Filing WO 2008/025839 of the applicant discloses a method for generating computer-generated video holograms in real time. Hologram values for the representation of a three-dimensional scene which is structured through object points on a spatial light modulator SLM are encoded based on image data with depth information. In analogy with the prior art solution mentioned above, the method disclosed in WO 2008/025839 is based on the idea not to reconstruct the object of the scene itself, but to reconstruct in one or multiple virtual observer windows the wave front which would be emitted by the object. A modulated wave field is generated from sufficiently coherent light by a spatial light modulator SLM, which is controlled by hologram values, and the desired real or virtual three-dimensional scene is reconstructed through interference in space. Virtual observer windows are generated in frustum-shaped reconstruction spaces with the SLM as a base. The windows are situated near the observer eyes and can be tracked to the actual observer position with the help of known position detection and tracking systems. The method disclosed in WO 2008/025839 is based on the fact that the region in which an observer sees a scene is defined by a frustum-shaped reconstruction space which stretches from the SLM to the observer window. The frustum can be approximated by a pyramid, because the observer window is much smaller than the SLM. Further, the method is based on the principle that the reconstruction of a single object point only requires a sub-hologram as a subset of the SLM. The information about each scene point is thus not distributed across the entire hologram, but is only contained in certain limited regions, the so-called sub-holograms. Following this concept, an individual object point of the scene is only reconstructed by a limited pixel region on the SLM, the so-called sub-hologram. The disclosure of WO 2008/025839 is based on the idea that for each object point the contributions of the sub-holograms to the entire reconstruction of the scene can be retrieved from look-up tables, and that these sub-holograms are accumulated so as to form a total hologram for the reconstruction of the entire scene.
According to a particularly preferred example of the method disclosed in WO 2008/025839, a view of the scene is defined by the position of each observer and their viewing direction. Each observer is assigned with at least one virtual observer window which lies near the observer eyes in an observer plane. In a preparatory process step the scene is discretised three-dimensionally into visible object points. These data may already be taken from an interface. The steps of the process disclosed in WO 2008/025839 are:
Step 1:
Finding the position of the sub-hologram for each object point: the position and extent of the corresponding sub-hologram are derived from the position of an object point, i.e. its lateral x, y coordinates and its depth distance.
Step 2:
Retrieval of the contributions of the corresponding sub-hologram from look-up tables.
Step 3:
Repetition of these two steps for all object points, where the sub-holograms are accumulated so as to form a total hologram for the reconstruction of the entire scene.
According to a simple example disclosed in WO 2008/025839, the size of a sub-hologram which is assigned to an object point is found based on the theorem of intersecting lines. The observer window or a part thereof which covers the pupils is projected through the object point into the hologram plane, i.e. on to the SLM. The indices of the pixels of the sub-hologram which are required to reconstruct this scene point are thus determined.
According to a further aspect of the disclosure of WO 2008/025839, additional corrective functions are applied to the sub-holograms or the total hologram, e.g. in order to compensate SLM tolerances which are caused by its position or shape, or to improve the reconstruction quality. The corrective values are for example added to the data values of the sub-holograms and/or of the total hologram. In addition, because every sub-hologram is defined by the actual position of the observer window, special look-up tables can be generated for more unusual observer windows, for example if the observer looks on the display at a large angle from a side position.
The principle of using look-up tables can preferably be extended, as described in WO 2008/025839. For example, parameter data for colour and brightness information can be stored in separate look-up tables. In addition, data values of the sub-holograms and/or the total hologram can be modulated with brightness and/or colour values from look-up tables. A colour representation is therein based on the idea that the primary colours can be retrieved from respective look-up tables.
The look-up tables on which the method disclosed in WO 2008/025839 is based are preferably generated in accordance with WO 2006/066906 or WO 2006/066919, which are filed by the applicant and are incorporated by reference. The look-up tables are then stored in suitable data carriers and storage media.
FIG. 26A illustrates the general idea of the disclosure of WO 2008/025839 with the example of a single observer. A view of a scene (S) is defined by the position and viewing direction of an observer (O). The observer is assigned with at least one virtual observer window (VOW) which lies near the observer eyes in a reference plane. A modulated wave field is generated from sufficiently coherent light by a spatial light modulator (SLM), which is controlled through hologram values. The method and the display derived from that method are based on the idea not to reconstruct the object of the scene itself, but to reconstruct in one or multiple virtual observer windows (VOW) the wave front which would be emitted by the object. In FIG. 26A, the object is represented by a single object point (PP). The observer (O) can watch the scene (S) through the virtual observer windows (VOW). The virtual observer windows (VOW) cover the eye pupils of the observer (O) and can be tracked to the actual observer position with the help of known position detection and tracking systems. Controlling the spatial light modulator (SLM) with the hologram values of the video holograms thereby causes the wave field, which is modulated in pixels and emitted from the display screen, to reconstruct the three-dimensional scene as desired by generating interference in the reconstruction space. As can be seen from FIG. 26A, according to the general principle of this implementation, a single object point (PP) of the scene (S) is only reconstructed by a limited pixel region on the spatial light modulator (SLM), the so-called sub-hologram (SH). As can be seen in FIG. 26A, according to a most simple solution, the size of a sub-hologram (SH) are defined based on the theorem of intersecting lines, whereby then the indices of the pixels required for the reconstruction of this object point (OP) are found. The position and extent of the sub-hologram (SH) are derived from the position of an object point (PP), i.e. its lateral x, y coordinates and its depth distance or z distance. Then, the hologram values required to reconstruct this point (PP) are now retrieved from the look-up table LUT.
The sub-hologram (SH) is modulated with a brightness and/or colour value and then accumulated into the hologram plane at the respective position so as to form a so-called total hologram. The data contained in the above-mentioned look-up tables are generated in advance. The data are preferably generated using the method described in WO 2006/066906, as cited in the prior art section above, and stored in suitable data carriers and storage media. With the help of the position and properties of the object points, the corresponding sub-holograms are computed in advance and the look-up tables of the sub-holograms, colour and brightness values and the corrective parameters are thus generated.
FIG. 26B illustrates this principle in more detail and shows the sub-holograms (SH1, SH2), which are assigned to the object points (P1, P2), respectively. It can be seen in FIG. 26B that these sub-holograms are limited and form a small and contiguous subset of the total hologram, i.e. the entire spatial light modulator (SLM). In addition to the position and extent of the sub-holograms which are determined based on the theorem of intersecting lines, as can be seen in FIG. 26, further functional relations are possible.
3. Discussion of Related Art
WO 2004/044659 (US2006/0055994) and U.S. Pat. No. 7,315,408B2, filed by the applicant, and incorporated herein in their entirety by reference, describe a device for reconstructing three-dimensional scenes by way of diffraction of sufficiently coherent light; the device includes a point light source or line light source, a lens for focusing the light and a spatial light modulator. In contrast to conventional holographic displays, the SLM in transmission mode reconstructs a 3D-scene in at least one ‘virtual observer window’ (see Appendix I and II for a discussion of this term and the related technology). Each virtual observer window is situated near the observer's eyes and is restricted in size so that the virtual observer windows are situated in a single diffraction order, so that each eye sees the complete reconstruction of the three-dimensional scene in a frustum-shaped reconstruction space, which stretches between the SLM surface and the virtual observer window. To allow a holographic reconstruction free of disturbance, the virtual observer window size must not exceed the periodicity interval of one diffraction order of the reconstruction. However, it must be at least large enough to enable a viewer to see the entire reconstruction of the 3D-scene through the window(s). The other eye can see through the same virtual observer window, or is assigned a second virtual observer window, which is accordingly created by a second light source. Here, a visibility region, which would typically be rather large, is limited to the locally positioned virtual observer windows. The known solution reconstructs in a diminutive fashion the large area resulting from a high resolution of a conventional SLM surface, reducing it to the size of the virtual observer windows. This leads to the effect that the diffraction angles, which are small due to geometrical reasons, and the resolution of current generation SLMs are sufficient to achieve a high-quality real-time holographic reconstruction using reasonable, consumer level computing equipment.
A mobile phone which generates a three dimensional image is disclosed in US2004/0223049, which is incorporated herein in its entirety by reference. However, the three dimensional image disclosed therein is generated using autostereoscopy. One problem with autostereoscopically generated three dimensional images is that typically the viewer perceives the image to be inside the display, whereas the viewer's eyes tend to focus on the surface of the display. This disparity between where the viewer's eyes focus and the perceived position of the three dimensional image leads to viewer discomfort after some time in many cases. This problem does not occur, or is significantly reduced, in the case of three dimensional images generated by holography.